Stereoscopic measurement system and method

ABSTRACT

A stereoscopic measurement system captures stereo images and determines measurement information for user-designated points within stereo images. The system comprises an image capture device for capturing stereo images of an object. A processing system communicates with the capture device to receive stereo images. The processing system communicates with the capture device to receive stereo images. The processing system displays the stereo images and allows a user to select one or more points within the stereo image. The processing system processes the designated points within the stereo images to determine measurement information for the designated points.

RELATED APPLICATIONS

This application claims priority in and is a continuation of U.S. patentapplication Ser. No. 12/125,809, entitled Stereoscopic MeasurementSystem and Method, the entire contents of which are incorporated hereinby reference. This application is related to co-pending, co-owned U.S.patent application Ser. No. 12/125,794, entitled StereoscopicMeasurement System and Method, filed on May 22, 2008; and U.S. patentapplication Ser. No. 12/125,801, entitled Stereoscopic MeasurementSystem and Method, filed on May 22, 2008; wherein the entire content ofeach application is incorporated herein by reference.

FIELD OF INVENTION Background of the Invention

Stereoscopic imaging, or stereoscopy, is used to obtainthree-dimensional information about an object based on a pair oftwo-dimensional images of that object. In general, stereoscopic imaginginvolves visually combining at least two images of an object, taken fromslightly different viewpoints, to produce the illusion ofthree-dimensional depth. By obtaining the two stereo images fromslightly different perspectives, coordinate locations of desiredmeasurement points identified in both images can be more accuratelydetermined.

Stereoscopic imaging is the basis for photogrammetry, which involvesproducing stereograms or a pair of stereo images of an object in orderto determine geometric properties and/or measurement information aboutthe object. Photogrammetry is used in various fields such asmanufacturing, architectural surveying, building preservation, andarchaeology in order to obtain measurement information for an object ofinterest. When obtaining measurements between particular measurementpoints on a desired object via photogrammetry, it is generally requiredthat the same measurement points are designated in both images to obtainaccurate measurement information.

With the advent of digital image sensors, computer-based imageprocessing techniques have been developed and applied to photogrammetry.However, the increase in digital image sensor resolution andadvancements in computer image-processing has not been efficientlyutilized for stereoscopic measurement purposes. Moreover, there is aneed for a stereoscopic processing system that allows a user to easilydesignate the same measurement points in stereo images of an object toobtain more accurate measurements.

SUMMARY OF THE INVENTION

According to one aspect, a system comprising modules executable with atleast one processor is provided for obtaining measurements of an object.The system comprises a memory storing a plurality of stereo images eachcomprising first and second images of the object. The system furthercomprises a composite module to combine at least two stereo images intoa composite stereo image, wherein the composite stereo image comprises acomposite first image and a composite second image, the composite firstimage comprises a composite of the first images of each of the at leasttwo stereo images, and the composite second image comprises a compositeof each of the second images of the at least two stereo images. Thesystem further comprises a user interface (UI) module to generate a listof stereo images for display. The UI module is further configured toreceive a first user input selecting the at least two stereo images fromthe list of stereo images, generate the first image and the second imageof each of the at least two stereo images for display, receive a seconduser input designating composite points in the first and second imagesof each of the at least two stereo images, generate the composite firstand second images for display based on the designated composite points,receive a third user input designating a first measurement point in thecomposite first image, receive a fourth user input designating a secondmeasurement point in the composite first image, receive a fifth userinput designating the first measurement point in the composite secondimage, and receive sixth user input designating a second measurementpoint in the composite second image. The system further comprises astereo point module to define a first stereo point that corresponds tothe first measurement point designated in the composite first and secondimages and to define a second stereo point that corresponds to thesecond measurement point designated in the composite first and secondimages. The system further comprises a cross measures module tocalculate the distance between the first stereo point and the secondstereo point.

According to another aspect, a system comprising modules executable withat least one processor is provided for obtaining measurements of anobject. The system comprises a memory storing a plurality of stereoimages each comprising first and second images of the object. The systemfurther comprises a composite module to combine at least two stereoimages of the plurality of stereo images into a composite stereo image.The composite stereo image comprises a composite first image and acomposite second image. The composite first image comprises a compositeof the first images of each of the at least two stereo images and thecomposite second image comprises a composite of each of the secondimages of the at least two stereo images. The system further comprises auser interface (UI) module to generate a list of stereo images fordisplay, to receive a first user input selecting the at least two stereoimages from the list of stereo images, and to generate the first imageand the second image of each of the at least two stereo images fordisplay. The UI module is further configured to receive a second userinput designating composite points in the first and second images ofeach of the at least two stereo images, to generate the composite firstand second images for display based on the designated composite points,to receive a third user input designating a first measurement point inthe composite first image, and to receive a fourth user inputdesignating a second measurement point in the composite first image. Thesystem further comprises a point selection module to identify a range ofpoints in the composite second image based on the first measurementpoint designated in the composite first image, to generate a selectionassist line in the composite second image based on the range of points,to identify another range of points in the composite second image basedon the second measurement point designated in the composite first image,to generate another selection assist line in the second image based onthe other range of points, to determine first pixel values adjacent tothe first measurement point designated in the composite first image, tocompare the determined first pixel values with other pixel values alongthe selection assist line to dynamically identify a corresponding firstmeasurement point in the composite second image with adjacent otherpixel values that match the determined first pixel values, to determinesecond pixel values adjacent to the second measurement point designatedin the composite first image, and to compare the determined second pixelvalues with second other pixel values along the other selection assistline to dynamically identify a corresponding second measurement point inthe second image with adjacent other pixel values that match thedetermined second pixel values. The system further comprises a stereopoint module to define a first stereo point that corresponds to thefirst measurement point designated in the composite first image andidentified in the composite second image and to define a second stereopoint that corresponds to the second measurement point designated in thecomposite first image and identified in the composite second image. Thesystem also comprises a cross measures module to calculate the distancebetween the first stereo point and the second stereo point.

According to another aspect, a method is provided for obtainingmeasurements from a stereo image of an object using at least oneprocessor. The stereo image comprising first and second images of theobject. The method comprises storing a plurality of stereo images eachcomprising first and second images of the object in a memory. The methodfurther comprises generating a list of the plurality of stereo imagesfor display. The method further comprises receiving a first user inputselecting at least two stereo images from the list. The method furthercomprises displaying the first image and the second image of each of theat least two stereo images. The method further comprises receiving asecond user input designating composite points in the first and secondimages of each of the at least two stereo images. The method furthercomprises combining the at least two stereo images into a compositestereo image based on the composite points, the composite stereo imagecomprising a composite first image and a composite second image. Themethod further comprises displaying the composite first image and thecomposite second image. The method further comprises receiving a thirduser input designating a first measurement point in the composite firstimage. The method further comprises receiving a fourth user inputdesignating a second measurement point in the composite first image. Themethod further comprises receiving a fifth user input designating thefirst measurement point in the composite second image. The methodfurther comprises receiving a sixth user input designating the secondmeasurement point in the composite second image. The method furthercomprises defining a first stereo point that corresponds to the firstmeasurement point designated in the composite first and second imagesand defining a second stereo point that corresponds to the secondmeasurement point designated in the composite first and second images.The method further comprises calculating the distance between the firststereo point and the second stereo point.

According to another aspect, a method is provided for obtainingmeasurements from a stereo image of an object using at least oneprocessor. The stereo image comprising first and second images of theobject. The method comprises storing a plurality of stereo images eachcomprising first and second images of the object in a memory. The methodfurther comprises generating a list of the plurality of stereo imagesfor display. The method further comprises receiving a first user inputselecting at least two stereo images from the list. The method furthercomprises displaying the first image and the second image of each of theat least two stereo images. The method further comprises receiving asecond user input designating composite points in the first and secondimages of each of the at least two stereo images. The method furthercomprises combining the at least two stereo images into a compositestereo image based on the composite points, the composite stereo imagecomprising a composite first image and a composite second image. Themethod further comprises displaying the composite first and secondimages. The method further comprises receiving a third user inputdesignating a first measurement point in the composite first image. Themethod further comprises receiving a fourth user input designating asecond measurement point in the composite first image. The methodfurther comprises identifying a range of points in the composite secondimage based on the first measurement point and identifying another rangeof points in the composite second image based on the second measurementpoint. The method further comprises generating a selection assist linein the composite second image based on the range of points andgenerating another selection assist line in the composite second imagebased on the other range of points. The method further comprisesdetermining first pixel values adjacent to the first measurement pointdesignated in the composite first image and determining second pixelvalues adjacent to the second measurement point designated in thecomposite first image. The method further comprises comparing thedetermined first pixel values with other pixel values along theselection assist line to dynamically identify a corresponding firstmeasurement point in the composite second image with adjacent otherpixel values that match the determined first pixel values and comparingthe determined second pixel values with second other pixel values alongthe other selection assist line to dynamically identify a correspondingsecond measurement point in the second image with adjacent other pixelvalues that match the determined second pixel values. The method furthercomprises defining a first stereo point that corresponds to the firstmeasurement point designated in the composite first image and identifiedin the composite second image. The method further comprises defining asecond stereo point that corresponds to the second measurement pointdesignated in the composite first image and identified in the compositesecond image. The method further comprises calculating the distancebetween the first stereo point and the second stereo point.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a stereoscopic measurement system inaccordance with an aspect of the present invention.

FIGS. 2A and 2B are perspective views of a stereo image capture deviceaccording to an aspect of the stereoscopic measurement system.

FIG. 3A is a block diagram of a stereoscopic measurement applicationaccording to one aspect of the stereoscopic measurement system.

FIGS. 3B-3D are image views of a camera sectioned for intrinsic cameracalibration.

FIG. 3E is an image of a vehicle with a central reference plane betweenselected points.

FIG. 3F is a geometric model for determining symmetry between selectedpoints on an image.

FIGS. 4A-4F are screen views of image management forms.

FIG. 5A is a geometric mapping model for a pinhole camera.

FIG. 5B is a three-dimensional model of the coordinate system for apinhole camera.

FIG. 6A-6B are triangulation models for determining the location of apoint in a coordinates system of an image capture device.

FIGS. 7A-7D are illustrations of an overlay process for creating acomposite stereo image pair from two stereo image pairs.

FIG. 8 is a flow chart illustrating a stereo image acquisition methodaccording to one aspect of the stereoscopic measurement system.

FIG. 9 is a flow chart illustrating a point measurement method within astereo image pair according to one aspect of the stereoscopicmeasurement system.

FIG. 10 is a flow chart illustrating a method for calculating andreporting measurements between designated measurement points in a stereoimage pair according to one aspect of the stereoscopic measurementsystem.

DETAILED DESCRIPTION

Aspects of the stereoscopic measurement system and method describedherein allow a user to generate stereo images of an object, to designatepoints within the stereo images of the object, and to obtain precisionmeasurements in reference to the designated points. One advantage of thesystem is the provision of a portable capture device that allows a userto capture stereo images of objects at remote locations. The portablecapture device transmits stereo images to a processing system to displaythe stereo images and to determine precision measurements betweendesignated points within the stereo images. Furthermore, the system canbe deployed in various environments, and is more portable and costeffective than conventional measuring systems.

FIG. 1 depicts an exemplary aspect of a stereoscopic measurement system100. The stereoscopic measurement system 100 enables a user 102 tocapture stereo images of an object 104 with a stereo image capturedevice 106. The stereo image capture device 106 comprises a left camera108 and a right camera 110. The left camera 108 and right camera 110are, for example, digital pinhole cameras located on opposing ends of aframe member 112.

A monitor 114 is centrally disposed between the left camera 108 and theright camera 110 on the frame member 112. The monitor 114 is configuredto display a left image 116 captured by the left camera 108 and a rightimage 118 captured by the right camera 110. Although a single monitor114 is depicted in FIG. 1, it is contemplated that separate monitors,such as depicted in FIGS. 2A and 2B, can be used to display the leftimage 116 and the right image 118.

Referring briefly to FIGS. 2A and 2B, aspects of an exemplary stereoimage capture device 106 are depicted. In this aspect, the stereo imagecapture device 106 is a portable hand-held apparatus that comprises abackbone 202 that is sufficiently rigid to limit flexing. For example,the backbone 202 can be constructed from a lightweight material, such asplastic or another suitable material.

A left pod 204 is affixed to the left end of the backbone 202 and aright pod 206 is affixed to the right end of the backbone 202. The leftpod 204 is configured to house the left camera 108, and the right pod206 is configured to house the right camera 110.

A hub 208 is located at the center of the backbone 202 and houses apower source (not shown) for powering the left and right cameras 108,110. For example, according to one aspect, the hub 208 comprises abattery compartment (not shown) that receives a battery. According toanother aspect, the hub 208 comprises power input terminals (not shown)configured to connect with a power cord that is connected to a poweroutlet.

According to another aspect, the hub 208 comprises a left monitor 210and a right monitor 212. The left monitor 210 and the right monitor 212are, for example, liquid crystal display (LCD) monitors. The leftmonitor 210 is connected to the left camera 108 and displays the leftimage 116. The right monitor 212 is connected to the right camera 110and displays the right image 118 of the object 104. The user 102maneuvers the stereo image capture device 106 to display left and rightimages 116, 118 of a desired portion of the object 104 via the left andright monitors 210, 212. The central location of the monitors 210, 212allows the user 102 to conveniently determine a common field of view forthe left and right cameras 108, 110.

A left handle 214 is located to the left of the hub 208 and a righthandle 216 is located to the right of the hub 208. Notably, it iscontemplated that the handles 214, 216 of the image capture device 106can be located in a different position or locations. The user 102 holdsthe image capture device 106 via the left handle 214 and right handle216. According to one aspect, the left handle 214 comprises a switch 218that controls the electronic shutters of the left camera 108 and theright camera 110. The switch 218 is wired to the left and right cameras108, 110 to ensure that the corresponding left and right images 116, 118are captured simultaneously. For example, when the left monitor 210 andright monitor 212 (or a single monitor 114) displays the left and rightimages 116, 118 of the desired area, the user 102 actuates or togglesthe switch 218 to capture the left and right images 116, 118.

According to one aspect, the left camera 108 and right camera 110 areconfigured to transfer images and image data to the hub 208 viauniversal serial bus (“USB”) cables. For example, the left camera 108 iswired to a communication port 220 by a USB cable, and the right camera110 is wired to the communication port 220 by another USB cable.

According to another aspect, the hub 208 is mounted on a swivel suchthat it can be rotated independently from the left camera 108 and theright camera 110. As a result, the user 102 can view the monitors 210,212 regardless of the orientation of the right and left cameras 108,110.

According to another aspect, lamps 222, 224 are located next to the leftand right cameras 108, 110. The purpose of the lamps 222, 224 is toilluminate the object 104 during capture of the left and right images116, 118. In one example, the lamps 222, 224 are configured to turn on,or flash, when the switch 218 is toggled. In another example, the lamps222, 224 are configured to turn on when a separate switch (not shown) istoggled.

Referring back to FIG. 1, the image capture device 106 is configured totransfer the left image 116 and the right image 118 to a processingsystem 120 for processing via a wired or wireless communication link.According to one aspect, the image capture device 106 is configured towirelessly transfer images to the processing system 120 in response tothe user 102 actuating a transmit switch (not shown) on the imagecapture device 106. In one example, a wireless transmitter 122 isconnected to the image capture device 106 via the communication port220. The transmitter 122 transmits a signal 124 comprising image datarepresentative of the left and right images 116, 118. Although thetransmitter 122 is depicted external to the image capture device 106, itis contemplated that the transmitter 122 may be integrated into theimage capture device 106.

A wireless receiver 126 is connected to the processing system 120 andreceives the signal 124 from the transmitter 122. The transmitter 122and corresponding receiver 126 may utilize a Gigabit Ethernet link, IEEE802.11 link, Ultra-Wide Band (UWB) link, or any other suitable wirelesscommunication link. The wireless transmitter 122 and wireless receiverare optional in some embodiments.

According to another aspect, the image capture device 106 transfers theleft image 116 and the right image 118 from the image capture device 106to the processing system 120 via a wired connection 128 in response tothe user 102 actuating the transmit switch (not shown). Alternatively,the processing system 120 automatically downloads images from thecapture device 106 in response to detecting the wired connection 128between the image capture device 106 and the processing system 120. Thewired connection 128 can be a USB connection, a FireWire connection, orany other suitable wired connection.

The processing system 120 comprises a stereoscopic measurementapplication (“measurement application”) 130. The measurement application130 comprises executable modules or instructions that enable theprocessing system 120 to process image data, display stereo images, andto obtain precise measurement data for designated points within stereoimages. In one aspect, the processing system 120 is a remote computer,such as a laptop computer or a personal computer station. In anotheraspect, the processing system 120 is a server computer.

A user interface (UI) 132 enables the user 102 to select images and/orto issue processing commands. Processing commands comprise, for example,commands to initiate image data acquisition from the image capturedevice 106 and/or commands to initiate image data analysis. In oneexample, the UI 132 comprises a display 134, such as a computer monitor,for viewing image data and an input device 136, such as a keyboard or apointing device (e.g., mouse, trackball, pen, touch pad, or otherdevice), for allowing the user 102 to interact with the image data.

The UI 132 is configured to display one or more input forms via thedisplay 134. The input forms enable the user 102 to select image datafor viewing and/or editing. The input forms also enable the user 102 todesignate points within stereo images and to display measurementinformation for the designated points.

According to one aspect, the processing system 120 comprises a memory138 for storing stereo image data for a particular object 104, includingprocessed and/or raw image data. For example, the memory 138 comprisesone or more files 140 each comprising processed and/or unprocessed imagedata for the object 104.

In one operational example, the stereoscopic measurement system 100compares user-designated points within stereo images of the object 104with known reference points for that object. By comparing user 102designated points within stereo images of an object 104, such as adamaged vehicle to corresponding reference points of an undamagedvehicle, the measurement system 100 determines one or more measurementsbetween the designated points and the reference points to quantify anamount of damage to the vehicle.

In another operational example, the stereoscopic measurement system 100detects a change in an object 104 that occurs over a period of time. Forexample, the stereoscopic measurement system 100 is used to calculate acurrent distance between two user-designated points in the stereo imagesof the exterior of a building. One of the designated points is, forexample, a reference point such as a ground elevation benchmark thatremains substantially constant over time. The other designated point is,for example, a target point on the exterior of the building. After aperiod of time has elapsed, the stereoscopic measurement system 100 isused to calculate the distance between the same reference point and thesame target point of the building. Accordingly, a change in thecalculated distance between the reference point and target pointindicates, for example, that the foundation of the building has shiftedand/or some other structural deviation has occurred.

Although the stereoscopic measurement system 100 is described herein asbeing used to obtain measurement data for vehicles and/or buildings, itis contemplated that the system 100 can be used to obtain measurementsfor any object 104 for which stereo images can be captured.

As another example, the stereoscopic measurement system 100 can be usedto catalog a three dimensional image of an artifact or personalproperty, such as a vase. For instance, the stereoscopic measurementsystem 100 is used to capture various stereoscopic images of the vase.Thereafter, measurements can be calculated between selected points onthe vase in all three dimensions. Thereafter, these measurements cancatalog and later used to verify the authenticity of the vase and/or togenerate a replica of the vase.

FIG. 3A depicts an exemplary stereoscopic measurement application 302(e.g., measurement application 130) according to one aspect of themeasurement system 100. The measurement application 302 comprisesmodules that enable the processing system 120 to process image data, togenerate stereo images, and to obtain precise measurements for userdesignated points within a generated stereo image.

A data-acquisition module 304 is configured to receive image data fromthe image capture device 106. For example, when the wired connection 128connects the image capture device 106 and the processing system 120, thedata acquisition module 304 detects the wired connection 128 andreceives the left and right images 116, 118 from the image capturedevice 106. As another example, when the left and right images 116, 118are being transferred to the processing system 120 via a wirelesscommunication, the data acquisition module 304 detects the wirelesscommunication from the image capture device 106 via the receiver 126 andreceives the left and right images 116, 118 from the image capturedevice 106. According to one aspect, the left and right images 116, 118images are deleted from the left and right cameras 108, 110 after beingtransferred to the processing system 120.

According to another aspect, the data acquisition module 304 isconfigured to retrieve intrinsic data 306 from the left and rightcameras 108, 110 for storage in the memory 138. As used herein,intrinsic data for a camera refers to geometric and opticalcharacteristics of the lens and the camera as determined via a cameracalibration process.

Camera calibration is the process of relating the ideal model of thecamera to the actual physical device and determining the position andorientation of the camera with respect to a world reference system.Stereoscopic calibration typically involves an internal or intrinsiccalibration process and an external or stereo calibration process. Asdescribed in more detail below, stereo calibration typically involvesdetermining the position and orientation of the left camera 108 andright camera 110 relative to each other with respect to a worldreference system.

The purpose of intrinsic calibration is to determine intrinsic data 306,such as lens distortion, focal length, and the principal point of animage for a particular camera. Intrinsic data 306 is determinedseparately for each of the left and right cameras 108, 110. According toone aspect, intrinsic calibration is performed during the final stagesof the manufacturing process of the image capture device 106. Forexample, after the image capture device 106 has been assembled and isoperable, intrinsic data 306 is determined separately for each of theleft camera 108 and right camera 110.

According to one aspect, the determined intrinsic data 306 for the leftcamera 108 is stored in a memory of the left camera 108, and thedetermined intrinsic data 306 for the right camera 110 is stored in amemory of the right camera 110. In one aspect, the determined intrinsicdata 306 is stored as XML files in the memory of each camera. Bydetermining intrinsic data 306 for each camera, the imperfections of apoint on an image can be effectively neutralized, thereby linking thepoint with the corresponding coordinates in the camera coordinatesystem.

According to one aspect, intrinsic data 306 is determined for each ofthe left and right cameras 108, by first capturing a series of photos ofa calibration image or jig 342 such as shown in FIGS. 3B-3D. Accordingto one aspect, the calibration image consists of alternating black andwhite squares or rectangles arranged in a planar checkerboard pattern.The series of photos are obtained for various orientations of thecalibration image 342.

In one example, the field of view of each camera, or image view space,344 is divided into nine sections (i.e., three rows and three columns).FIG. 3B depicts the calibration image 342 in a first orientationpositioned in a section of the image view space 344 that corresponds tothe top row and the left column. Images of the calibration image 342 inthe first orientation are captured in each of the nine sections by eachcamera. FIG. 3C depicts the calibration image 342 in a secondorientation (e.g., rotated approximately forty-five degrees). Images ofthe calibration image 342 in the second orientation are captured in eachof the nine sections by each camera. FIG. 3D depicts the calibrationimage 342 in a third orientation (e.g., tilted backward approximatelyforty-five degrees). Images of the calibration image 342 in the thirdorientation are captured in each of the nine sections by each camera.

The dimensions of the individual checker patterns are known. As aresult, the camera intrinsic values of focal length, lens distortion,and principal point location can be determined. For example, imageprocessing techniques are used to identify the corners of each square inthe checkerboard and construct perspective lines connecting thesecorners. If the perspective lines are slightly curved instead ofstraight, a formula can be derived to straighten their curviness andused thereafter to remove image distortions. As a result, the formulacan be used to establish a mapping of world straight lines to imagestraight lines. In one example, this formula is a row vector of scalarvalues representing lens distortion and the misalignment of the opticalaxis center of the image plane, called the principal point, to themechanical axis of the image plane. The two corners along any edge of asquare in the checkerboard correspond to pixels representing thesecorners on the image plane. Homogeneous vectors drawn from the imagesensor cross at the focal point and pass through the corners of thesquare of known size. The focal length is determined as the height ofthe triangle formed by these two lines from the image plane to theplanar checkerboard pattern.

According to another aspect, the data acquisition module 304 isconfigured to determine if the intrinsic data 306 retrieved from theleft camera 108 and right camera 110 has been updated before storing theintrinsic data 306 in the memory 138. For example, when the intrinsicdata 306 is stored as an XML file, the data acquisition module 304compares XML file metadata, such as a creation date and time associated,with XML files being retrieved from each camera, with similar XML filemetadata associated with XML files previously stored in the memory 138.If XML file metadata associated with XML files being retrieved from theleft camera 108 and right camera 110 indicates that the creation dateand time for those XML files was created after XML files previouslystored in the memory 138, the data acquisition module 304 replaces thepreviously stored XML files with the XML files being retrieved from theleft camera 108 and right camera 110.

According to another aspect, a pairing module 308 pairs the left image116 and the right image 118 to create a stereo image pair 310. Thepairing module 308 then stores the stereo image pair 310 andcorresponding download history data 312 in the memory 138. The downloadhistory data 312 comprises, for example, a time and date that the imagedata from the left and right cameras 108, 110 included in the stereoimage pair 310 were transferred from the image capture device 106 to theprocessing system 120. According to another aspect, the download historydata 312 comprises metadata for each of the left and right cameras 108,110. Metadata identifies, for example, a camera model, a film type, andleft or right camera.

An image-processing module 314 processes the stereo image pair 310 todetermine if the left and right images 116, 118 are images of acalibration image 342. For example, the image-processing module 314employs a pattern recognition algorithm to detect the known geometricalpattern of the calibration image 342 in the stereo image. If theimage-processing module 314 determines a particular stereo image pair310 comprises images of a calibration image 342, a stereo calibrationmodule 316 is executed.

The stereo calibration module 316 is configured to determine stereocalibration data 318 for the image capture device 106. For example, thestereo calibration module 316 determines the pinhole locations for theleft and right cameras 108, 110 relative to a common element within acalibration pattern (e.g., calibration image 342) to establish areference origin for a coordinate system that corresponds to the imagecapture device 106. In another aspect, the stereo calibration module 316determines the separation distance between the center of the pinholelocations for the left and right cameras 108, 110 and the angularpositioning of each of the cameras in relation to the image capturedevice 106. The determined pinhole locations for the left and rightcameras 108, 110, the separation distance, and the angular position ofleft and right cameras 108, 110 are referred to collectively as stereocalibration data 318. In one aspect, stereo calibration data is amatrix, either called the essential matrix or the fundamental matrix,comprising both translation and rotation values describing the stereocalibration data 318. The stereo calibration module 316 stores thestereo calibration data 318 in the memory 138. The stereo calibrationdata 318 is used to triangulate the exact location of user-designatedpoints within a stereo image pair 310.

According to one aspect, stereo calibration is performed just prior tocapturing images of a particular object 104 for which measurementinformation is desired. Environmental conditions, such as temperatureand humidity levels, can affect the shape of the image capture device106 (e.g., material contraction and expansion), and, thus, affect thepositioning of the cameras 108, 110 relative to each other. Byperforming stereo calibration prior to capturing images of a desiredobject 104, the stereo calibration data 318 can be determined based onthe most current positioning of the cameras 108, 110 relative to eachother.

According to one aspect, stereo calibration involves using a calibrationimage (e.g., calibration image 342) to determine the current position ofthe left and right cameras 108, 110 relative to each other. For example,the image capture device 106 captures left and right images 116, 118 ofthe calibration image. The size of the individual checker patterns inthe image, the focal length of the cameras, principal point, and lensdistortion are known parameters. As a result, the separation distanceand/or angular position between the left and right cameras can bedetermined by applying triangulation techniques to selected points inthe left and right images. Triangulation is described in more detailbelow in reference to FIGS. 6A and 6B.

According to another aspect of the stereoscopic measurement system 100,the image-processing module 314 associates the stereo calibration data318 with a stereo image pair 310 based on the download history data 312.For example, a stereo image pair 310 that has a transfer date and timethat is subsequent to the date and time associated with a particularstereo image pair 310 in which the calibration image 342 was detected,is associated with the stereo calibration data 318 determined from thatparticular stereo image pair 310.

A user interface (UI) module 320 is configured to generate an imagemanagement form 322 for the display via the UI 132. In one example, theUI module 320 retrieves the stereo image pair 310 from the memory 138and allows the user 102 to interact with the left and right images 116,118 included in the stereo image pair 310 via the image management form322 on the display 134. The image management form 322 comprises variousviews that allow a user to display image data, to interact with imagedata, and to specify points within a stereo image pair 310 formeasurement.

FIGS. 4A-4D depict various screen views of an image management form 322displayed on the display 134. In one aspect, the user 102 interacts withthe image management form 322 depicted in FIG. 4A via an input device(e.g., input device 136) to display an existing project. As used herein,the term “project” refers to a file that comprises one or more stereoimage pairs 310. For example, the user 102 uses the input device 136 toselect an open project control 402 on the image management form 322 todisplay a list of existing projects, such as depicted in FIG. 4B.Thereafter, the user 102 selects a particular project from the list ofexisting projects to open using standard file opening techniques.

According to another aspect, the user 102 uses the input device 136 tointeract with the image management form 322 to display a list of stereoimages pairs 406 included in the selected project. For example, the user102 uses the input device 136 to select a project images control 404 todisplay the list of stereo images pairs 406 included in the selectedproject.

According to another aspect, the user 102 uses the input device 136 tointeract with the image management form 322 to delete one or more stereoimages from the list of stereo images pairs 406 included in a project.For example, the user 102 uses the input device 136 to enable or selecta check box control 408 adjacent to a stereo image pair 310. Thereafter,the user 102 uses the input device 136 to select, for example, a deletecontrol 410 to permanently delete the selected stereo image pair 310from memory 138. In another example, the user 102 uses the input device136 to select, for example, a remove control 412 to remove the selectedstereo image pair 310 from the project, but not from the memory 138.

According to another aspect, the user 102 interacts with the imagemanagement form 322 to add one or more new stereo images pairs to anexisting project. For example, the user 102 uses the input device 136 toselect a new images tab 414, such as shown in FIG. 4C, to display a listof new stereo image pairs 416. In one example, the user 102 selects astereo image pair 310 from the list of new stereo image pairs 416 byusing the input device 136 to enable or select a check box 418 adjacenta desired new stereo image pair 310. Thereafter, the user 102 uses theinput device 136 to select, for example, an add control 420 to add theselected stereo image pair 310 to the existing project.

According to another aspect, the user 102 interacts with the imagemanagement form 322, such as depicted in FIG. 4C, to create a newproject. For example, the user 102 uses the input device 136 to select anew project control 422 on the image management form 322 to display thelist of new stereo image pairs 416. The user 102 then uses the inputdevice 136 to select one or more stereo image pairs 310 from the list ofnew stereo image pairs 416 to include in the new project. For example,the user 102 uses the input device 136 to enable or select the check box418 adjacent the desired new stereo image pair 310. Thereafter, the user102 uses the input device 136 to select the add control 420 to add theselected stereo image pair 310 to the new project.

According to another aspect, the user 102 interacts with the imagemanagement form 322, such as depicted in FIG. 4C, to delete one or morestereo image pairs from the list of new stereo image pairs 416. Forexample, the user 102 uses the input device 136 to enable or select thecheck box 418 adjacent to a desired new stereo image pair 310.Thereafter, the user 102 uses the input device 136 to select, forexample, a delete control 424 to delete the selected stereo image pair310 from the list of new stereo images 416.

According to another aspect, the user 102 interacts with the imagemanagement form 322 to select a particular stereo image pair 310 withina particular project for viewing. For example, the user 102 uses theinput device 136 to enable the check box control 408 (see FIG. 4A)adjacent to a stereo image pair 310 included in the list of stereoimages 406 for an existing project. As another example, the user 102uses the input device 136 to enable the check box 418 (see FIG. 4C)adjacent to a stereo image pair 310 included in the list of new stereoimages 416 for a new project.

The UI module 320 generates the selected stereo image pair 310 fordisplay in a left image window 426 and a right image window 428 of theimage management form 322 in response to the users' selection. Inparticular, the left image window 426 displays the left image 116 of thestereo image pair 310 and the right image window 428 displays the rightimage 118 of the stereo image pair 310.

According to another aspect, the UI module 320 displays the left image116 or the right image 118 in an active window 430 in response to theuser 102 selecting the left image window 426 or the right image window428. For example, the user 102 uses the input device 136 to select theleft image window 426 to display the left image 116 in the active window430 or to select the right image window 428 to display the right image118 in the active window 430. Notably, the stereo image pair 310displayed in FIG. 4C comprises left and right images 116, 118 of acalibration image 342.

According to another aspect, the user 102 interacts with the imagemanagement form 322 to designate one or more measurement points withinan image displayed in the active window 430. For example, the user 102selects either the left image window 426 or the right image window 428to display the corresponding left image 116 or right image 118 in theactive window 430. The user 102 then uses the input device 136 to panacross and/or zoom in and out of the image displayed in the activewindow 430. In one example, the selected image window (e.g. left imagewindow 426 or right image window 428) that corresponds to the image(e.g. left image 116 or right image 118) displayed in the active window430 comprises a focus rectangle 434, such as shown in FIG. 4E. The focusrectangle 434 outlines the portion of the image visible in the activewindow 430. The user 102 can pan the image in the active window 430 byusing the scroll bars 436 adjacent to the active window 430.Alternatively, the user 102 pans the image in the active window 430 bydragging the focus rectangle 434 by, for example, positioning a mousepointer over the focus rectangle 434, pressing and holding the mousebutton while the focus rectangle 434 is moved to the desired location.

After the user 102 visually locates the desired measurement point, theuser 102 interacts with the image in the active window 430 to select thepoint. In one example, the user 102 positions a mouse pointer over thedesired location and clicks the mouse button to designate the point. Inresponse to a point designation by the user 102, the UI module 320displays a precision mark 438 at the location on the image displayed inthe active window 430 where the user designate the point.

According to another aspect, the user 102 interacts with the imagedisplayed in the active window 430 to fine-tune the location of thedesignated point. For example, the user uses arrow keys of a keyboard toadjust the location of the point.

In order to obtain precise measurements, the user 102 must designate thesame measure points in both the left image 116 and right image 118 ofthe stereo image pair. Therefore, after designating the desired point ina first image (e.g. left image 116) of the stereo image pair 310, theuser 102 selects the other image window (e.g. right image window 428) todisplay the second image (e.g. right image 118) of the stereo image pair310 in the active window 430. The user 102 then designates the samepoint in the second image being displayed in the active window 430. Inresponse to the user's point designation, the UI module 320 displaysanother precision mark 440 at the location on the second image displayedin the active window 430 where the user designated the same point. Inother words, the user 102 selects common points in both of the left andright images 116, 118 of the stereo image pair 310.

Referring back to FIG. 3A, a point selection module 324 is configured toassist a user 102 select the same point in the right image 118 byautomatically identifying a range of points in the right image 118 thatcorrespond to the point designated by the user 102 in the left image116. As described above, left camera 108 and right camera 110 are, forexample, pinhole cameras.

FIG. 5A depicts the pinhole model of a camera. An optical axis 502extends in the view direction of the camera. All projection lines, orhomogeneous vectors, of an image pass through a pinhole 504 of thecamera. An image plane 506 is where a particular point (P.sub.1) 508 inthe three dimensional world (X, Y, Z) is projected through the pinhole504 of the camera. For example, a projection vector 510 or line frompoint P.sub.1 508 will pass through the pinhole 504 onto the image plane506 of the camera at a point P.sub.2 512. The distance between thepinhole 504 and the image plane 506 along the optical axis 502 is thefocal length, f, of the camera.

FIG. 5B depicts a three-dimensional coordinate system for the pinholemodel used as the basis for single-camera and stereoscopic mathematics.Place the pinhole 504 of the camera (e.g., left camera) at the origin Oof the coordinate system, and the image plane 506 parallel to the XYplane of the coordinate system. The relation between the threedimensional world coordinates of point P.sub.1 508 and the coordinateson the image plane (x, y) can be expressed by the following:

x=f*X/Z  (1);

y=f*Y/Z  (2);

where f is the focal length of the lens. Thus, the homogeneous vector510 defines a point on the image plane of the camera.

Referring back to FIG. 3A, the point selection module 324 defines arange of possible matching points in the right image 118 based on adesignated point in the left image 116. According to one aspect, thepoint selection module 324 uses the series of points defined by ahomogeneous vector (e.g., projection vector 510) in FIG. 5B from adesignated point in the left image 116 along with intrinsic calibrationdata and stereo calibration data for the left camera 108 and the rightcamera 110 to define a range of possible matching points in the rightimage 118. As described above, intrinsic calibration data comprisesfocal lengths, principal points, and lens distortions for the leftcamera 108 and right camera 110 and stereo calibration data includes therelative translation and rotation of the left camera 108 and rightcamera 110.

According to another aspect, the point selection module 324 generates aselection line 441, such as depicted in FIG. 4D, on the right image 118when displayed in the active window 430. The selection line 441corresponds to the range of possible points in the right image 118 thatcorrespond to the point designated in the left image 116.

According to another aspect, the point selection module 324 isconfigured to automatically identify a point in the right image 118 thatcorresponds to the point designated by the user in the left image 116.For example, in addition to generating a selection line 441 in the rightimage 118, the point selection module 324 utilizes a pattern recognitionalgorithm to identify a point along the selection line 441 thatcorresponds to the point designated by the user 102 in the left image116. For example, the point selection module 324 determines the value ofeach pixel adjacent to the point selected by the user 102 in the leftimage 116.

Digital images are comprised of pixels, and each pixel has a value thatrepresents a grayscale value or color value. In grayscale images, thepixel value is a single number that represents the brightness of thepixel. The most common pixel format is the byte image, where this numberis stored as an 8-bit integer giving a range of possible values from 0to 255. Typically, a pixel value of zero is taken to be black, and apixel value of 255 is taken to be white. Values in between make up thedifferent shades of gray. In color images, separate red, green, and bluecomponents must be specified for each pixel (assuming an RGB colorspace). In other words, the pixel value is actually a vector of threenumbers. The three different components can be stored as three separategrayscale images known as color planes (one for each of red, green andblue), which can be recombined when displaying or processing.

The point selection module 324 then compares the determined values ofthe pixels adjacent to the point selected by the user in the left image116 to identify a particular point that has adjacent pixels withmatching values along the selection line 441 in the right image 118. TheUI module 320 displays the other precision mark 440 at the location inthe right image 118 that corresponds to same point designated in theleft image 116.

The user 102 repeats the point selection process to define a secondmeasurement point in each of the right and left images 116, 118. Forexample, the user 102 selects the left image window 426 to display theleft image 116 in the active window 430, and then uses the input device136 to perform pan and/or zoom operations to locate a desired secondmeasurement point in the left image 116. After the user visually locatesthe second measurement point, the user 102 uses the input device 136 todesignate the location of the second point in the left image 116 asdescribed above in reference to the first measurement point. In responseto the user's second point designation, the UI module 320 displays aprecision mark 442 at the designated location in the left image 116.

The user 102 then interacts with the image management form 322 todesignate the same second measurement points in the right image 118. Forexample, the user 102 selects the right image window 428 to display theright image 118 in the active window 430. The user 102 uses the inputdevice 136 to designate the location of the same second measurementpoints in the right image 118.

Alternatively, the user uses the input device 136 to designate thelocation of the same second measurement points in the right image 118along another selection line (not shown) generated in the right image118. The other selection line is generated by the point selection module324 and corresponds to the range of possible points in the right image118 that correspond to the second measurement point. In another aspect,the user 102 relies on the point selection module 324 to automaticallylocate the same second measurement point in the right image 118. The UImodule 320 displays a precision mark 444 at the location in the rightimage 118 that corresponds to same point designated in the left image116.

A stereo point module 326 uses triangulation to define a stereo point inthe virtual three-dimensional coordinate system of the image capturedevice 106 based on the common points designated in both the left image116 and right image 118 of the stereo image pair 310. In other words, astereo point or three dimensional position of a designated point can bereconstructed from the perspective projections of that point on theimage planes of the left and right cameras 108, 110 once the relativeposition and orientation of the two cameras are known. The stereo pointcorresponds to the x, y, z coordinate values of the common designatedpoint in the left and right images 116, 118 as determined fromtriangulation.

FIG. 6A depicts an epipolar triangulation model for determining thelocation of a point P.sub.1 602 in a coordinate system of the imagecapture device 106. The left camera 108 and the right camera 110 areeach pinhole cameras with parallel optical axes. For purposes ofillustration assume that the left camera 108 and right camera 110 eachhave the same focal length F 604. Further, assume that the center ofleft camera 108 is located at X.sub.1 606 along the X-axis and that thecenter of the right camera 110 is located at X.sub.2 608 along theX-axis. The distance (D) 610 between the centers of each lens (i.e.,center of pinholes) is equal to the difference between X.sub.1 606 andX.sub.2 608. In this example, the optical axis of each camera is in theXZ plane and the XY plane is parallel to the image plane of both theleft and right cameras 108, 110. Assume that the X axis is the baselineand the origin, O, of the coordinates system (X, Y, Z) of the imagecapture device 106 is located at the lens center (e.g., pinhole) of theleft camera 108. The three dimensional coordinates of the point P.sub.1602 can be determined from the following algorithms:

Define a scaling factor as:

S=D/|x1−x2|  (3).

Then, the X, Y, Z coordinates can be determined as follows:

z=f*S  (4);

X=x1*S  (5); and

Y=y1*S=y2*S  (6).

FIG. 6B depicts another epipolar triangulation model for determining thelocation of a point P.sub.1 602 in a coordinate system of the imagecapture device 106. The left camera 108 and the right camera 110 areeach pinhole cameras angled with their optical axes toed in toward eachother. For purposes of illustration assume that the left camera 108 andright camera 110 each have the same focal length F 604. The distancebetween the origins of each camera's pinhole model is represented bytranslation vector t. Any rotation, including the toe-in of the opticalaxes, can be represented by a rotation matrix R. A mapping of the leftand right camera coordinate systems will bind projection vectorsrepresenting point P1 into one overall coordinate system. One suchmapping is the essential matrix, E, resulting from the product of theskew-symmetric matrix of vector t, as indicated by reference character612, and the rotation matrix R, as indicated by reference character 614.Projection vectors x1 and x2 are now related in a single coordinateframe as:

x1*E*x2=0  (7).

Coordinates (X, Y, and Z) of point P1 are derived from simpletriangulation of these projection vectors within the combined coordinateframe.

A cross measure module 328 calculates the distance between two or morestereo points defined by the stereo point module 326. In one example,the cross measure module 328 calculates the distance between two or morestereo points in response to a user selecting a measure control 446,such as shown in FIG. 4E. The UI module 320 displays the calculateddistance in a measurement table 448.

A composite module 330 is configured to combine or stitch two stereoimage pairs 310 into a composite stereo image pair 332. The compositestereo image pair 332 comprises two stereo image pairs 310 in whichthere is some overlap between the right and left images 116, 118included in each of the two stereo image pairs 310. By combining twosuch stereo image pairs 310, measurements can be obtained between afirst point in the left and right images 116, 118 of a first stereoimage pair image and a second point in the left and right images 116,118 of a second stereo image pair. In particular, measurement can beobtained between the non-overlapping portions of the right and leftimages 116, 118 included in the two stereo image pairs 310.

According to one aspect, the user 102 defines composite points in eachof two stereo image pairs 310 and overlays the two stereo image pairs310 based on the composite points to create the composite stereo imagepair 332. For example, the users uses the point selection techniquesdescribed above to select the same three non-co-linear and uniquelyidentifiable reference points in both of the stereo image pairs 310. Thecomposite module 330 overlays to the two stereo image pairs 310 suchthat the three non-co-linear and uniquely identifiable reference pointsmatch to create the composite stereo image pair 332 in response to theuser 102 selecting a create composite control 450, such as shown in FIG.4A. The composite stereo image pair 332 comprises a composite left imageand a composite right image. The composite module 330 then stores thecomposite stereo image pair 332 in the memory 138.

FIGS. 7A-7C depict an overlay process for creating a composite stereoimage pair 332 based on two stereo images of a vehicle 702. Although theoverlay process involves combining both left and right images from twostereo pairs, for purposes of illustration the overlay process isdescribed in reference to combining the left images 116 of two stereopairs 310. FIG. 7A depicts a first left image 704 of a first stereoimage pair that corresponds to a front section of the vehicle 702.

FIG. 7B depicts a second left image 706 of a second stereo image pair310 that corresponds to the mid-section of the vehicle 702. As describedabove, the user 102 uses the point selection techniques described aboveto select the same three non-co-linear and uniquely identifiablereference points in both the first and second left images. In thisexample, reference points 708, 710, 712 are selected in both the firstand second left images 704, 706.

FIG. 7C depicts an overlay of the first left image pair 704 and secondleft image 706 such that reference points 708, 710, 712 match to createa composite left image 714. As shown in FIG. 7D, a first measurementpoint 716 can be selected in the front section of the vehicle 702 and asecond measurement point 718 can be selected in the mid-section of thevehicle 702 via the composite left image 714.

Notably, a same overlay process is used to create a composite rightimage based on a first right image of the first stereo image pair thesecond right image of the second stereo image pair.

According to another aspect, the user 102 interacts with the imagemanagement form 322 to add the composite stereo image pair 332 to anexisting project. For example, the user 102 uses the input device 136 toselect, for example, the add control 420 (see FIG. 4C) to add thecomposite stereo image pair 332 to the existing project.

According to another aspect, the user 102 interacts with the imagemanagement form 322 to select a composite stereo image pair 332 todisplay the left images and right images 116, 118 of each stereo pair310 included in the composite stereo image pair 332. In one example, theuser 102 selects a composite stereo image pair 332 for viewing by usingthe input device 136 to enable or select a check box (not shown)adjacent to a desired composite stereo image pair 332. The UI module 320displays images from the left and right images 116, 118 for each of thestereo images in image windows 452-458 in response to the user selectingthe composite stereo image pair 332.

According to another aspect, the user 102 uses the input device 136 toselect one of image windows 452-458 to display the corresponding imagein the active window 430.

Referring back to FIG. 3A, the measurement application 302 is configuredto retrieve information from a measurement database 334 that comprisesstereo point data 336 for specific defined points on one or more objects104. In one example, the measurement database 334 comprises stereo pointdata 336 for defined stereo points, or reference stereo points, along avehicle body for a specific type of vehicle when the body is notdamaged.

By comparing stereo point data from the measurement database 334 tostereo points generated based on user-designated points in stereo imagesof a vehicle of the same type with body damage, a precise assessment ofthe amount of damage to the vehicle can be determined. For example, thedistance between a reference stereo point on an undamaged vehicle can becompared to stereo points defined based on corresponding user-designatedpoints in stereo images of a damaged vehicle. The distance between thereference stereo point and one or more defined stereo points can bemeasured to determine an amount of damage to the vehicle.

As another example, by comparing stereo point data 336 from themeasurement database 334 to stereo points generated based onuser-designated points in stereo images of an undamaged vehicle,deviations in the body of the undamaged vehicle can be identified. As aresult, the measurement system 100 can be used to verify that products,such as vehicles, are being manufactured within desired tolerances.Although the measurement database 334 is depicted as being external theprocessing system 120, it is contemplated that the measurement database334 may be located on the processing system.

A symmetry module 338 is configured to determine if there are symmetrydeviations between selected points on an object. According to oneaspect, using the techniques described above, the user 102 opens a newproject or an existing project that comprises at least two stereo imagepairs that show opposing sides of an object. The user 102 then uses thepoint selection techniques described above to define a set of stereopoints on each opposing side of the object 104.

For example, if the object 104 is a vehicle, the user 102 selects a setof points (e.g., first and second points) in a first stereo image pair310 comprising left and right images 116, 118 of a passenger side of thevehicle. The user 102 then selects another set of points (e.g., firstand second points) in a second stereo image pair 310 comprising left andright images 116, 118 of a driver side of the vehicle. The userinteracts with the image management form 322 to define point details fora selected set of points. For example, the user 102 uses the inputdevice 136 to select, for example, a point detail control 462 to displaya point detail table 464, such as depicted in FIG. 4F. The user 102 thendesignates one set of points as a reference set by using the inputdevice 136 to enable an adjacent check box control 466.

According to one aspect, the symmetry module 338 is configured to definea central reference plane 350 based on the designated reference set inresponse to the user selecting a symmetry control 468, such as depictedin FIG. 4C. As an example, FIG. 3E depicts a top view of a vehiclehaving a first point and a second point 354 selected on the passengerside 356 a corresponding first point 358 and a corresponding secondpoint 360 point selected on a driver side 362. Assuming the userdesignates the first point 352 and second point 354 selected on thepassenger side 356 as the reference set, the symmetry module 338 definesthe central reference plane 350 between the first point 352 and thesecond point 354.

According to one aspect, symmetry deviations are determined anddisplayed as deviation values via the image management form. In oneexample, the determined deviation values are displayed as two values,one for distance from the center plane (Y) and one for the combined Xand Z values.

FIG. 3F depicts a geometrical model for determining symmetry between afirst set of points on a first side of an object and a second set ofpoints on a second side. For purposes of illustration, the geometricalmodel will be described in reference to the example shown in FIG. 3E. Avector 362 is defined between the first and second points 352, 354 and amidpoint 364 of the vector 362 is determined. The center reference plane350 is defined as the plane that passes though the midpoint 364 and thatis perpendicular to the vector 362. The midpoint 364 is also defined asthe origin of an X, Y, and Z coordinate system.

The distance X.sub.11 from the first point 352 to a perpendicular pointon the reference plane 350 is determined and the distance X.sub.12 fromthe second point 354 to the perpendicular point on the reference plane350 is determined. The distance X.sub.21 from the corresponding firstpoint 358 to a perpendicular point on the reference plane 350 isdetermined and the distance X.sub.22 from the corresponding second point360 to the perpendicular point on the reference plane 350 is determined.Corresponding distances are compared to determine symmetry deviationvalues. For example, distance X.sub.11 is compared to distance X.sub.21.According to one aspect, the measurement application 130 defines thedifference in distances as the X deviation error. If neither point is areference point, the measurement application 130 divides the X deviationerror. If at least one point is a reference point, the measurementapplication 130 assigns the X deviation error to the non-referencepoint.

According to another aspect, the measurement application 130 determinesthe points at which the first point 352 and second point 354 projectsinto the reference plane 350, and determines the points at which thecorresponding first point 358 and second point 360 projects into thereference plane 350. The measurement application 130 determines acombined YZ error of the first and second points 352, 354 as a functionof the distance between the projected points from the passenger side356. Similarly, the measurement application 130 determines the combinedYZ error of the corresponding first and second points 358, 360 as afunction of the distance between the projected points from the driverside 362. If neither point is a reference point, the measurementapplication 130 splits the YZ error. Otherwise, the measurementapplication 130 assigns the YZ error to the non-reference point.

According to another aspect, a reporting module 340 creates customizedreports. In one example, the reports include the results of thecalculations of cross measures based on user-designated points. Theresults can be displayed in a tabular format on the image managementform 334. In another example, the reports comprise deviations fromsymmetry or comparative measurements based on stereo point dataretrieved from the measurement database 330. In another example, imagesand/or diagrams are incorporated into reports. For example, if theobject 104 being analyzed is a vehicle, the reports may include imagesor diagrams 470 of the vehicle with measure points identified andlabeled, such as depicted in FIG. 4E. Notably, reports can be generatedfor display and can optionally be printed and/or saved to disk.

According to another embodiment, the measurement application 130 isexecuted on a server computer, and reports and/or image data can becommunicated to remote computers, such as personal computers, laptops,personal digital assistants, and any other computing device via acommunication network, such as the Internet, an Intranet, or any othersuitable communication network.

Computer readable media 370 may include volatile media, nonvolatilemedia, removable media and non-removable media, may also be anyavailable medium that may be accessed by the general purpose computingdevice. By way of example and not limitation, computer readable media370 may include computer storage media and communication media. Computerstorage media may further include volatile, nonvolatile, removable, andnon-removable media implemented in any method or technology for storageof information such as computer readable instructions, data structures,program modules, or other data. Communication media may typically embodycomputer readable instructions, data structures, program modules, orother data in a modulated data signal, such as a carrier wave or othertransport mechanism and include any information delivery media. Thoseskilled in the art will be familiar with the modulated data signal,which may have one or more of characteristics set or changed in such amanner that permits information to be encoded in the signal. Wiredmedia, such as a wired network or direct-wired connection, and wirelessmedia, such as acoustic, radio frequency, infrared, and other wirelessmedia contemplated by the stereoscopic measurement system 100, areexamples of communication media discussed above. Combinations of any ofthe above media are also included within the scope of computer readablemedia discussed above.

FIG. 8 illustrates a stereo image acquisition method according to anaspect of the measurement system. At 802, the image capture device 106captures the left image 116 and right image 118 of the object 104 viathe left camera 108 and the right camera 110, respectively. Acommunication link is established between the processing system 120 andthe image capture device 106 at 804. As described above, thecommunication link can be established via a wired connection 128 or thecombination of a wireless transmitter 124 and wireless receiver 126.

At 806, the measurement application 130 is executed in response to theestablished communication link between the processing system 120 and theimage capture device 106. The measurement application 130 retrieves theleft and right images 116, 118 and downloads intrinsic data from theleft and right cameras at 808. At 810, the measurement application 130pairs the left image 116 and the right image 118 to create the stereoimage pair 310. The measurement application 130 stores the stereo imagepair 310 and corresponding download history data 312 in the memory 138at 812. As described above, the download history data 312 comprises, forexample, a time and date that the left image 116 and the right image 118of the stereo image pair 310 were transferred from the image capturedevice 106 to the processing system 120.

FIG. 9 illustrates a point measurement method within a stereo image pair310 according to one aspect of the measurement system 100. At 902, themeasurement application 130 displays an image management form 322 on thedisplay 134 that allows a user to select a stereo image pair 310 forviewing. The left image 116 and right image 118 of the selected stereoimage pair 310 in the left image window 426 and the right image window428 at 904. At 906, the left image 116 or the right image 118 isdisplayed in the active window 430 in response to the user 102 selectingthe left image window 426 or the right image window 428. As describedabove, the user 102 uses the input device 136 to select the left imagewindow 426 to display the left image 116 in the active window 430 or toselect the right image window 428 to display the right image 118 in theactive window 430.

At 908, the user 102 interacts with the image management form 322 todesignate two measurement points within a first image of the stereoimage pair that is displayed in the active window 430. For example,after the user 102 visually locates the desired point, the user 102positions a mouse pointer over the desired location in the first imageand clicks the mouse button to designate two measurement points in thefirst image. Precision marks (e.g., precision marks 438, 442) aredisplayed at the locations in the first image displayed in the activewindow 430 where the user designated the point at 910.

At 912, the user 102 interacts with the image management form 322 viathe input device 136 to designate the same measurement points within thesecond image of the stereo image pair 310 displayed in the active window430. Optionally at 914, the measurement application 130 displays aselection line that defines a range of possible matching points in thesecond image 116 based on each of the user designated points in thefirst image. At 916, the user 102 interacts with the image managementform 322 to designate the same measurement points along the selectionlines within the second image of the stereo image pair 310 displayed inthe active window 430.

As another option, at step 918, the measurement application 130automatically identifies points in the second image that corresponds tothe points designated by the user in the first image. As describe above,in addition to generating selection lines 438 in the second image 116,the measurement application utilizes a pattern recognition algorithm toidentify a point along the selection lines that correspond to the pointsdesignated by the user 102 in the first image. At 920, precision marks(e.g., precision marks 440, 444) are displayed at locations in thesecond image that correspond where the user 102 designated measurementpoints in the second image at 912 or 916, or where the measurementapplication 130 automatically identified the matching measuring pointsin the second image at 918.

FIG. 10 illustrates a method for calculating and reporting measurementsbetween designated measurement points according to one aspect of themeasurement system 100. At 1002, the measurement application 130 definesa first stereo point for the first measurement point designated in theleft image 116 and the right image 118. The measurement application 130defines a second stereo point for the second measurement pointdesignated in the left image 116 and the right image 118 at 1004. Asdescribed above, each stereo point corresponds to the x, y, zcoordinates of the common designated point in the left and right images116, 118 as determined from triangulation. The distance between thefirst and second measurement points is calculated as function of thecoordinate values of the first and second stereo points at step 1006. Atstep 1008, the calculated distances are displayed to the user via theimage management form. At step 1010, the reports are generated inresponse to input received from a user via the image management form.

When introducing elements of aspects of the invention or the embodimentsthereof, the articles “a,” “an,” “the,” and “said” are intended to meanthat there are one or more of the elements. The terms “comprising,”“including,” and “having” are intended to be inclusive and mean thatthere may be additional elements other than the listed elements.

As various changes could be made in the above constructions, products,and methods without departing from the scope of aspects of theinvention, it is intended that all matter contained in the abovedescription and shown in the accompanying drawings shall be interpretedas illustrative and not in a limiting sense.

1. A system for obtaining measurements of an object, the systemcomprising at least one processor, wherein the processor is configuredto: store a plurality of stereo images each comprising first and secondimages of the object; combine at least two stereo images into acomposite stereo image, wherein the composite stereo image comprises acomposite first image and a composite second image, the composite firstimage comprises a composite of the first images of each of the at leasttwo stereo images, and the composite second image comprises a compositeof the second images of each of the at least two stereo images;designate composite points in the first and second images of each of theat least two stereo images; designate a first measurement point and asecond measurement point in the composite first image; designate thefirst measurement point and a second measurement point in the compositesecond image; define a first stereo point that corresponds to the firstmeasurement point designated in the composite first and second imagesand to define a second stereo point that corresponds to the secondmeasurement point designated in the composite first and second images;and calculate the distance between the first stereo point and the secondstereo point.
 2. The system of claim 1 wherein the processor is furtherconfigured to: generate the first image and the second image of each ofthe at least two stereo images for display; generate a list of stereoimages for display; select the at least two stereo images from the listof stereo images; and generate the composite first and second images fordisplay based on the designated composite points.
 3. The system of claim1 further comprising a memory, wherein the processor is furtherconfigured to store the plurality of stereo images in the memory.
 4. Thesystem of claim 1 wherein the plurality of stereo images are receivedfrom an image capture device.
 5. The system of claim 4 wherein the imagecapture device comprises a first camera and a second camera.
 6. Thesystem of claim 5 wherein the memory is configured to store downloadhistory data for each of the first and second images.
 7. The system ofclaim 6 wherein the download history data comprises metadata andintrinsic calibration data for the first and second cameras and a timeand date that the first and second images included in each stereo imagewere received from the image capture device.
 8. The system of claim 5wherein the processor is further configured to: process each of theplurality of stereo images to determine if a particular first image anda particular second image of a particular stereo image include images ofa calibration pattern; determine stereo calibration data for the imagecapture device when the particular first and second images of theparticular stereo image are of the calibration pattern, the stereocalibration data comprising location information for the first camerarelative to the second camera in a coordinate system of the imagecapture device; and store the stereo calibration data.
 9. The system ofclaim 8 wherein the processor is further configured to associate thestereo calibration data with another particular stereo image of theplurality of stereo images based on the download history for that otherparticular stereo image when the first and second images of that otherparticular stereo image are not of the calibration pattern.
 10. Thesystem of claim 1 further comprising a calibration pattern, wherein aparticular first image and a particular second image of a particularstereo image each includes an image of the calibration pattern.
 11. Thesystem of claim 1 wherein the processor is further configured to createa customized report comprising the calculated distance between the firststereo point and the second stereo point.
 12. The system of claim 11further comprising reference stereo point data, and a measurementdatabase, wherein the measurement database is configured to store thereference stereo point data; and wherein the reference stereo point datacorrespond to at least one reference stereo point on each of theplurality of objects; and wherein the processor is further configuredto: create the customized report comprising calculated distancesselected from a group consisting of a first distance between the firststereo point and the second stereo point, a second distance between thefirst stereo point and the reference stereo point, and a third distancebetween the second stereo point and the reference stereo point.
 13. Thesystem of claim 1 wherein the composite points comprise at least threecorresponding reference points in each of the first and second images ofthe at least two stereo images, and wherein the processor is furtherconfigured to: overlay each of the first images included in the at leasttwo stereo images such that the three corresponding reference pointsmatch to create the composite first image; and overlay each of thesecond images included in the at least two stereo images such that thethree corresponding reference points match to create the compositesecond image.
 14. The system of claim 1 wherein the composite pointscomprise at least three corresponding reference points in each of thefirst and second images of the at least two stereo images, wherein theprocessor is further configured to: overlay each of the first imagesincluded in the at least two stereo images such that the threecorresponding reference points match to create the composite firstimage; and overlay each of the second images included in the at leasttwo stereo images such that the three corresponding reference pointsmatch to create the composite second image.
 15. A method for obtainingmeasurements of an object using at least one processor, the methodcomprising: combining at least two stereo images into a composite stereoimage, wherein the composite stereo image comprises a composite firstimage and a composite second image, the composite first image comprisesa composite of the first images of each of the at least two stereoimages, and the composite second image comprises a composite of thesecond images of each of the at least two stereo images; designatingcomposite points in the first and second images of each of the at leasttwo stereo images; designating a first measurement point and a secondmeasurement point in the composite first image; designating the firstmeasurement point and a second measurement point in the composite secondimage; defining a first stereo point that corresponds to the firstmeasurement point designated in the composite first and second imagesand to define a second stereo point that corresponds to the secondmeasurement point designated in the composite first and second images;and calculating the distance between the first stereo point and thesecond stereo point.
 16. The method of claim 15, wherein the designatedcomposite points comprise at least three corresponding reference pointsin each of the first and second images of the at least two stereoimages, the method further comprising; overlaying each of the firstimages included in the at least two stereo images such that the threecorresponding reference points match to create the composite firstimage; and overlaying each of the second images included in the at leasttwo stereo images such that the three corresponding reference pointsmatch to create the composite second image.
 17. A method for obtainingmeasurements of an object for retrieval, wherein the object has anidentification, the method comprising: providing a stereo image of anobject, the stereo image comprising a first image and a second image ofan object; defining a stereo point in the stereo image; creating anassociation between the defined stereo point with the identification ofthe object; and storing data comprising the identification of theobject, the defined stereo point, and the association in a memory. 18.The method of claim 17 further comprising a data repository, and whereinthe memory is part of the data repository.
 19. The method of claim 18further comprising retrieving the stored data from the data repository,the stored data comprising the identification of the object, the definedstereo point, and the association.
 20. The method of claim 17 furthercomprising a data repository, and wherein the stored data is transferredfrom the memory to the data repository.
 21. The method of claim 17further comprising a database, and wherein the memory is part of thedatabase.
 22. The method of claim 17 further comprising a database, andwherein the stored data is transferred from the memory to the database.23. The method of claim 17 wherein: the object is a vehicle; and theidentification comprises the year, make, model, and trim line of thevehicle.
 24. The method of claim 17 wherein: the object is a part of avehicle; and the identification comprises the year, make, model, andtrim line of the vehicle, and a part description of the part of thevehicle.
 25. The method of claim 24 wherein the identification furthercomprises a part number of the part of the vehicle.
 26. The method ofclaim 17 further comprising: creating an association between the stereoimage pair with the identification of the object; and storing datacomprising the stereo image and the association in the memory.
 27. Themethod of claim 26 further comprising retrieving the stored data fromthe data repository, the stored data comprising the identification ofthe object, the stereo image, the defined stereo point, the associationbetween the defined stereo point with the identification of the object,and the association between the stereo image pair with theidentification of the object.
 28. The method of claim 27 furthercomprising: retrieving stereo data so that at least two stereo imagesare retrieved; combining the at least two stereo images into a compositestereo image, wherein the composite stereo image comprises a compositefirst image and a composite second image, the composite first imagecomprises a composite of the first images of each of the at least twostereo images, and the composite second image comprises a composite ofthe second images of each of the at least two stereo images; designatingcomposite points in the first and second images of each of the at leasttwo stereo images; designating first measurement point and a secondmeasurement point in the composite first image; designating the firstmeasurement point and a second measurement point in the composite secondimage; defining a second stereo point, the second stereo pointcorresponding to the first measurement point designated in the compositefirst and second images and to define a third stereo point, the thirdstereo point corresponding to the second measurement point designated inthe composite first and second images; and calculating the distancebetween the second stereo point and the third stereo point.
 29. Themethod of claim 17 further comprising retrieving the stored data fromthe memory, the stored data comprising the identification of the object,the defined stereo point, and the association.
 30. A system forobtaining measurements of an object for retrieval, wherein the objecthas an identification, the system comprising: a stereo image of anobject, the stereo image comprising a first image and a second image ofan object; a memory; a processor configured to: define a stereo point inthe stereo image; create an association between the defined stereo pointwith the identification of the object; and to store data comprising theidentification of the object, the defined stereo point, and theassociation in the memory.
 31. The system of claim 30 further comprisinga data repository, and wherein the memory is part of the datarepository.
 32. The system of claim 31 wherein the processor is furtherconfigured to retrieve the stored data from the data repository, thestored data comprising the identification of the object, the definedstereo point, and the association.
 33. The system of claim 30 furthercomprising a data repository, and wherein the processor is furtherconfigured to transfer the stored data from the memory to the datarepository.
 34. The system of claim 30 further comprising a database,and wherein the memory is part of the database.
 35. The system of claim30 further comprising a database, and wherein the processor is furtherconfigured to transfer the stored data from the memory to the datarepository.
 36. The system of claim 30 wherein: the object is a vehicle;and the identification is the year, make, model, and trim line of thevehicle.
 37. The system of claim 30 wherein: the object is a part of avehicle; and the identification comprises the year, make, model, andtrim line of the vehicle, and a part description of the part of thevehicle.
 38. The system of claim 37 wherein the identification furthercomprises a part number of the part of the vehicle.
 39. The system ofclaim 30 wherein the processor is further configured to: creating anassociation between the stereo image with the identification of theobject; and storing data comprising the stereo image and the associationin the memory.
 40. The system of claim 39 wherein the processor isfurther configured to retrieve the stored data from the memory, thestored data comprising the identification of the object, the stereoimage pair, the defined stereo point, the association between thedefined stereo point with the identification of the object, and theassociation between the stereo image with the identification of theobject.
 41. The system of claim 39 wherein the processor is furtherconfigured to retrieve the stored data from the data repository, thestored data comprising the identification of the object, the stereoimage pair, the defined stereo point, the association between thedefined stereo point with the identification of the object, and theassociation between the stereo image with the identification of theobject.
 42. The system of claim 41 wherein the processor is furtherconfigured to: retrieve stored data so that at least two stereo imagesare retrieved; combine at least two stereo image into a composite stereoimage, wherein the composite stereo image comprises a composite firstimage and a composite second image, the composite first image comprisesa composite of the first images of each of the at least two stereo imagepairs, and the composite second image comprises a composite of thesecond images of each of the at least two stereo image pairs; designatecomposite points in the first and second images of each of the at leasttwo stereo image pairs; designate a first measurement point and a secondmeasurement point in the composite first image; designate the firstmeasurement point and a second measurement point in the composite secondimage; define a second stereo point, the second stereo pointcorresponding to the first measurement point designated in the compositefirst and second images and to define a third stereo point, the thirdstereo point corresponding to the second measurement point designated inthe composite first and second images; and calculate the distancebetween the second stereo point and the third stereo point.
 43. Thesystem of claim 30 wherein the processor is further configured toretrieve the stored data from the memory, the stored data comprising theidentification of the object, the defined stereo point, and theassociation.